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Executive  Summary 


Background 

Over  the  last  several  years  the  Navy  has  invested  considerable  time  and  energy 
in  developing  a  human  resource  strategy  that  emphasizes  the  link  between 
personnel  quality  and  operational  readiness.  Success  will  require  the  linkage  of 
human  resource  approaches  with  the  accomplishment  of  organizational  goals; 
implementation  of  recruiting,  selection,  training,  and  retention  approaches  that 
foster  mission  accomplishment;  and  transformation  to  a  results-oriented 
organizational  culture. 

One  human  resource  component  that  will  play  a  vital  role  is  personnel 
advancement.  The  process  of  identifying  potential  candidates  for  advancement, 
and  selecting  those  to  be  promoted  is  critical  for  an  organization's  continuing 
success  and  productivity.  Organizations  use  a  variety  of  different  personnel 
advancement  systems  to  facilitate  the  decision-making  process,  and  many  of 
these  systems  incorporate  multiple  criteria  to  improve  accurate  evaluation  of 
candidates. 

For  Navy  Enlisted  Sailors,  the  current  advancement  system  provides  for  the 
orderly  progression  of  qualified  personnel  to  higher  levels  of  responsibility 
throughout  their  naval  careers.  Generally,  advancement  is  based  on 
demonstrated  proficiency  in  assigned  duties,  on  the  performance  evaluation  and 
recommendation  of  the  commanding  officer,  and  on  written  examinations. 

Approach 

While  the  current  advancement  system  is  operating  adequately,  it  is  prudent  to 
speculate  on  how  Navy  Enlisted  advancement  might  operate  within  a  Eive 
Vector  Model  (5VM)  perspective.  It  is  possible  to  envision  development  of  an 
advancement  algorithm  that  links  performance  across  all  five  vectors  to 
advancement  to  the  next  paygrade.  The  notion  is  that  an  advancement  score 
could  be  computed  based  on  achievement  of  defined  milestones  across  all 
vectors,  as  part  of  the  broader  plan  to  define  the  career  paths  associated  with  a 
member's  professional  development,  personal  development,  leadership  abilities, 
certifications  and  qualifications,  and  overall  performance. 

The  first  step  in  developing  such  a  system  would  be  to  establish  a  scientifically 
sound  approach  for  weighting  the  contribution  of  each  of  the  five  vectors  in 
making  advancement  decisions.  In  2003  we  conducted  a  research  study  with  that 
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objective  in  mind.  This  feasibility  study  used  a  policy  capturing  methodology  to 
assist  the  judgmental  processes  related  to  advancement  decisions. 

The  feasibility  study  involved  conducting  workshops  where  participants  were 
presented  with  5V1VI  profiles,  with  preset  "scores"  on  the  different  vectors  and 
asked  to  rate  the  promotability  of  each  "profiled  Sailor."  Analyses  of  data  from 
these  workshops  essentially  determined  the  relative  weight  of  each  vector  in  the 
advancement  algorithm.  These  relative  weights  were  generated  for  advancement 
to  apprentice,  journeyman,  and  master  skill  levels. 

While  the  feasibility  study  established  that  a  policy  capturing  methodology 
provides  a  scientifically  sound  procedure  for  determining  5V1VI  advancement 
weights,  it  emphasized  a  long-range  perspective,  where  vectors  were  defined 
based  on  an  anticipated,  fully  developed  5VM.  The  purpose  of  the  current  study 
was  to  apply  a  more  specific,  near-term  focus,  where  definitions  of  the  five 
vectors  reflected  currently  available  quantitative  advancement  data  organized 
within  a  5VM  framework. 

Once  again,  we  applied  a  policy  capturing  methodology  to  determine  vector 
weights  across  the  three  skill  levels.  In  all,  24  workshops  were  conducted  in 
Bremerton,  WA,  Millington,  TN,  Norfolk,  VA,  and  Pensacola,  FL,  to  capture  the 
vector  priorities  of  the  347  participants. 

Results  and  Discussion 

The  basic  finding  was  that  job  performance  is  overall  the  most  important  factor, 
but  as  skill  level  progresses,  leadership  becomes  increasingly  important,  to  a 
point  where  performance  and  leadership  are  roughly  equally  important. 
Professional  development  is  also  an  important  factor,  especially  early  in  a  Navy 
career,  and,  certifications/ qualifications  have  some  importance  at  the  lower  and 
middle  advancement  levels.  Finally,  personal  development  was  afforded  lesser 
importance  for  the  advancement  of  Enlisted  personnel. 

The  policy  capturing  research  described  here  provided  a  scientifically  sound 
approach  for  pooling  the  judgment  and  wisdom  of  experienced  Enlisted 
personnel  regarding  the  relative  weights  that  should  be  placed  on  each  of  the 
5VM  vectors  in  making  advancement  decisions.  This  study  provided  a  way  for 
the  Eleet  to  collectively  provide  their  judgment  about  advancement  policy  in  the 
U.S.  Navy. 
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Introduction 


Over  the  last  several  years  the  Navy  has  invested  considerable  time  and  energy 
in  developing  a  human  resource  strategy  that  emphasizes  the  link  between 
personnel  quality  and  operational  readiness.  Certainly,  a  key  factor  in  the  success 
of  the  Navy's  human  resource  management  strategy  is  the  sustained  attention  of 
its  senior  leaders.  Most  recently,  this  'commitment  to  its  people'  from  Navy 
leadership  was  underscored  by  release  of  ADM  Mullen's  CNO  Guidance  for 
2006,  which  called  for  building  upon  this  strategy,  to  "deliver  a  transformed, 
competency-focused  manpower  and  personnel  system  for  all  segments  of  our 
workforce"  (p.  7;  Mullen,  2006).  Success  will  require  the  linkage  of  human 
resource  approaches  with  the  accomplishment  of  organizational  goals; 
implementation  of  recruiting,  selection,  training,  and  retention  approaches  that 
foster  mission  accomplishment;  and  transformation  to  a  results-oriented 
organizational  culture. 


Five  Vector  Model 

Many  of  these  human  resource  initiatives  view  a  Sailor's  career  as  comprised  of 
five  distinct  areas  or  "vectors."  These  vectors  include:  professional  development, 
personal  development,  leadership,  certifications  and  qualifications,  and  job 
performance.  This  five  vector  model  (5VM)  forms  the  foundation  around  which 
the  Navy  identifies  the  knowledge,  skills,  and  abilities  that  Sailors  need  to 
succeed  in  today's  Navy.  A  description  of  how  this  5VM  is  conceptualized  can  be 
found  in  Table  1. 


Navy  Advancement 

One  human  resource  component  that  will  play  a  vital  role  is  personnel 
advancement.  The  process  of  identifying  potential  candidates  for  advancement, 
and  selecting  those  to  be  promoted  is  critical  for  an  organization's  continuing 
success  and  productivity.  Organizations  use  a  variety  of  different  personnel 
advancement  systems  to  facilitate  the  decision-making  process,  and  many  of 
these  systems  incorporate  multiple  criteria  to  improve  accurate  evaluation  of 
candidates. 


Table  1.  Vector  Definitions 


Professional  Development:  This  vector  addresses  Enlisted  rating  training.  It  focuses  on  a  Sailor's  ability 
to  acquire  job  knowledge  and  skills  through  such  sources  as  formal  schools,  correspondence  courses  and 
on-the-job  training.  This  is  the  vector  that  will  show  what  must  be  known,  and  what  level  of  expertise  is 
expected  of  an  individual  as  a  professional  in  a  particular  career  field.  As  much  as  possible,  this  vector  will 
incorporate  industry  standards,  providing  the  training  and  credentials  recognized  by  civilian  counterparts. 

Personal  Development:  This  vector  involves  such  areas  as  general  military  training  (GMT)  and  financial 
planning  and  management,  as  well  as  college-level  courses  that  allow  Sailors  to  complete  degree 
programs.  It  focuses  on  "life  skills"  needed  outside  the  workplace.  Personal  Development  focuses 
attention  on  an  individual's  development  as  a  person,  not  just  as  a  Sailor.  The  notion  here  is  to  give 
Sailors  opportunities  to  develop  themselves  as  people  by  providing  tools  for  personal  growth.  Seven 
clearly  defined  competency  clusters  comprise  the  personal  development  vector:  values,  lifelong  learning, 
life  skills,  financial  management  skills,  health,  wellness  and  recreation,  and  interpersonal  relations. 

Leadership:  This  vector  focuses  on  providing  Sailors  with  the  tools  and  critical  thinking  skills  they  will 
need  as  leaders.  The  focus  is  on  developing  the  ability  of  an  individual  to  accomplish  the  mission  as  well 
as  mentor  and  develop  others.  Leadership  addresses  leadership  training,  providing  the  tools  and  critical 
thinking  skills  needed  to  be  effective  leaders.  It  emphasizes  the  importance  of  receiving  effective 
leadership  training  throughout  a  career,  not  just  at  given  stages  of  advancement.  Components  of  the 
Leadership  vector  include:  communication,  professionalism,  responsibility/authority/accountability, 
subordinate  development,  and  leadership  awareness. 

Certifications  and  Qualifications:  This  vector  focuses  on  unit-level  requirements  and  related  industry 
certifications  that  are  directly  tied  to  job  proficiencies.  It  may  include  industry-recognized  standards  such 
as  airframe  and  power  plant  licenses  in  the  aviation  community,  Microsoft  certifications  for  those  in 
information  technology,  and  merchant  marine  qualifications  and  licenses  for  deck  ratings.  For  example,  an 
IT  would  receive  a  Microsoft  Network  Certification  or  a  Construction  Mechanic  might  earn  an  ASE 
certification  after  displaying  a  given  level  of  proficiency  within  his  or  her  rating. 

Performance:  This  vector  assesses  an  individual's  overall  job  performance.  As  the  Sailor  achieves 
specific  milestones,  he/she  will  get  promoted  -  as  long  as  his/her  performance  measures  up.  The 
Performance  vector  takes  into  account  all  issues  related  to  job  performance,  including  task  proficiency 
and  productivity,  adaptability,  initiative,  knowledge  and  support  of  unit/command  objectives,  problem 
solving  and  decision  making,  communication  skills  and  work  ethic.  It  is  the  measurement  of  a  Sailor's 
overall  on-the-job  performance. 


For  Navy  Enlisted  Sailors,  the  current  advancement  system  provides  for  the 
orderly  progression  of  qualified  personnel  to  higher  levels  of  responsibility 
throughout  their  naval  careers.  Generally,  advancement  is  based  on 
demonstrated  proficiency  in  assigned  duties,  on  the  performance  evaluation  and 
recommendation  of  the  commanding  officer,  and  on  written  examinations. 


5VM  Advancement 

While  the  current  advancement  system  is  operating  adequately,  it  is  prudent  to 
speculate  on  how  Navy  Enlisted  advancement  might  operate  within  a  5VM 
perspective.  It  is  possible  to  envision  development  of  an  advancement  algorithm 
that  links  performance  across  all  five  vectors  to  advancement  to  the  next 
paygrade.  The  notion  is  that  an  advancement  score  could  be  computed  based  on 


achievement  of  defined  milestones  across  all  vectors,  as  part  of  the  broader  plan 
to  define  the  career  paths  associated  with  a  member's  professional  development, 
personal  development,  leadership  abilities,  certifications  and  qualifications,  and 
overall  performance. 

This  model  could  then  indicate  the  advancement  potential  for  recruit,  apprentice, 
journeyman,  and  master  skill  levels;  yielding  data  to  communicate  the 
advancement  potential  of  an  individual  to  both  the  member  and  promotion 
boards.  The  new  scoring  system  would  translate  individual  performance  in  these 
five  vectors  into  an  overall  ranking,  similar  to  the  final  multiple  produced  after 
today's  Navy-wide  examinations.  Such  a  system  could  provide  a  much  more 
comprehensive  and  responsive  way  of  looking  at  who  is  'fully  qualified,'  and 
identify  them  as  the  individuals  who  should  advance. 


Study  1:  2003 

The  first  step  in  developing  such  a  system  would  be  to  establish  a  scientifically 
sound  approach  for  weighting  the  contribution  of  each  of  the  five  vectors  in 
making  advancement  decisions.  In  2003  we  conducted  a  research  study  with  that 
objective  in  mind.  This  feasibility  study  used  a  policy  capturing  methodology  to 
assist  the  judgmental  processes  related  to  advancement  decisions. 

A  rich  literature  has  developed  surrounding  the  use  of  a  policy-capturing 
methodology  to  address  how  raters  use  information  to  make  judgments.  Policy 
capturing  has  been  used  to  examine  issues  relevant  to  performance  appraisal 
(e.g.,  Naylor  &  Wherry,  1965;  Zedeck  &  Kafry,  1977),  individual  motivation  (e.g., 
Dachler  &  Mobley,  1973),  job  analysis  (Sanchez  &  Levine,  1989),  job  evaluation 
(e.g..  Madden,  1964),  and  managerial  promotion  (Stumpf  &  London,  1981).  A 
policy  capturing  approach  requires  each  decision  maker  to  rate  a  large  number  of 
individuals  who  vary  along  several  criteria.  These  criteria  may  be  considered 
predictors  of  the  decision,  and  the  decision  maker's  policy  is  inferred  through 
statistical  analysis  of  the  ratings. 

The  feasibility  study  involved  conducting  workshops  where  participants  were 
presented  with  5VM  profiles,  with  preset  "scores"  on  the  different  vectors  and 
asked  to  rate  the  promotability  of  each  "profiled  Sailor."  Analyses  of  data  from 
these  workshops  essentially  determined  the  relative  weight  of  each  vector  in  the 
advancement  algorithm.  These  relative  weights  were  generated  for  advancement 
to  apprentice,  journeyman,  and  master  skill  levels. 

In  2003,  workshops  were  conducted  at  Navy  sites  across  the  continental  United 
States,  to  capture  the  policies  of  the  participants.  In  all,  89  Sailors  participated  in 
the  study;  27  Sailors  participated  in  the  recruit  to  apprentice  workshops,  32 
Sailors  participated  in  the  apprentice  to  journeyman  workshops,  and  30  Sailors 
participated  in  the  journeyman  to  master  workshop. 


Table  2  presents  the  results  of  the  feasibility  study  for  Enlisted  advancement.  As 
the  table  indicates,  job  performance  is  overall  the  most  important  factor,  but  as 
rank  progresses,  within  the  Enlisted  ranks,  leadership  becomes  increasingly 
important,  to  a  point  where  performance  and  leadership  are  roughly  equally 
important.  Professional  development  is  also  an  important  factor,  especially  early 
in  a  Navy  career.  And,  certif ications/qualifications  have  some  importance  at  the 
lower  and  middle  advancement  levels.  Einally,  personal  development  was 
afforded  lesser  importance  for  the  advancement  of  Enlisted  personnel  (for  more 
detailed  information  see  Borman,  Hedge,  Bruskiewicz,  and  Bourne,  2003). 


Table  2.  Relative  Weights  for  Enlisted  Personnel  Advancement 


Vector 

Recruit  to 
Apprentice 
(Percent) 

Apprentice  to 
Journeyman 
(Percent) 

Journeyman  to 
Master  (Percent) 

Professional  Development 

17.78 

10.62 

9.33 

Personal  Development 

1.98 

3.47 

2.60 

Leadership 

12.26 

31.26 

41.23 

Certifications/Qualifications 

9.99 

10.05 

6.09 

Performance 

57.99 

44.60 

40.75 

Objectives  of  Study  2 

While  the  feasibility  study  provided  a  useful  roadmap  for  how  advancement 
might  proceed  within  a  Sea  Warrior  5V1VI  framework,  to  realize  this  Eleet  vision 
of  advancement,  each  of  the  vectors  would  need  to  identify  or  develop  measures 
of  performance  that  can  be  employed  to  measure  degrees  of  success  on  the 
vector.  When  these  scoring  systems  are  developed  within  each  vector,  overall 
advancement  scores  can  be  computed  using  the  policy  capturing  results. 

In  reality,  however,  presently  the  performance  requirements  for  most  of  the  five 
vectors  are  not  sufficiently  established,  nor  are  they  sufficiently  populated  with 
data  to  proceed  with  the  strategy  envisioned  in  the  feasibility  study.  So,  although 
the  feasibility  study  established  that  a  policy  capturing  methodology  provides  a 
scientifically  sound  procedure  for  determining  5VM  advancement  weights,  the 
study  was  based  on  a  broad,  long-range  perspective.  The  purpose  of  the  current 
study  was  to  apply  a  more  specific,  near-term  focus. 

In  other  words,  rather  than  ask  the  Eleet  to  speculate  on  the  importance  of  each 
of  the  five  vectors  in  an  anticipated,  fully  developed  5VM,  as  was  the  case  in  the 
feasibility  study,  it  would  also  be  informative  to  understand  how  each  of  the 
vectors  might  be  viewed  as  contributing  to  a  Sailor's  "promotability"  in  today's 
Navy.  In  order  to  do  so,  the  objectives  of  the  present  study  required  new 


operational  definitions  of  the  five  vectors  that  reflected  currently  available 
advancement  data  organized  within  a  5VM  framework.  Once  the  vectors  were 
re-defined,  we  could  then  proceed  to  apply  the  policy  capturing  methodology  to 
determine  vector  weights  across  the  three  skill  levels.  Such  a  design  would  also 
allow  us  to  compare  the  results  of  the  preliminary  weighting  study  with  those  of 
the  current  study. 


Methodology  for  Study  2;  2006 


Overview 

We  applied  the  same  methodology  briefly  discussed  in  the  feasibility  study 
mentioned  earlier.  This  policy  capturing  methodology  refers  to  a  set  of 
procedures  used  to  describe  statistically  the  unique  information  processing 
strategies  of  raters  (Bottenberg  &  Christal,  1961;  Christal,  1968). 

The  policy  capturing  approach  consists  of  several  primary  components, 
including:  (a)  the  presentation  to  raters  of  a  series  of  profiles  composed  of  scores 
on  a  number  of  stimulus  cues;  (b)  rater  review  and  rating  of  each  profile  of 
information  on  some  overall  criterion  which  summarizes  the  information 
available;  and  (c)  the  use  of  multiple  regression  analysis  within  the  framework  of 
the  general  linear  model  to  calculate  the  extent  to  which  the  overall  ratings  are 
predictable  given  the  profile  scores,  and  to  compute  the  relative  importance  or 
contribution  of  each  of  these  individual  cues  in  determining  the  overall  ratings. 

The  statistical  equation  resulting  from  the  regression  analysis  is  thought  to  depict 
the  "captured  rating  policy"  for  each  rater.  In  other  words,  the  regression 
equation  represents  an  explicit  depiction  of  the  way  in  which  dimensional 
information  is  weighted  and  combined  to  arrive  at  an  overall  promotability 
rating. 


Procedure 

In  the  current  study,  we  applied  the  policy  capturing  methodology  to  gather  the 
perspectives  of  a  cross-section  of  Navy  personnel  concerning  how 
accomplishments  in  each  of  the  five  vectors  contribute  to  overall  Enlisted 
promotability.  One  hundred  and  twenty  profiles  were  randomly  generated  to 
depict  varying  levels  of  performance/ accomplishment  on  each  of  the  five 
vectors.  After  these  profiles  were  generated,  project  staff  reviewed  them  to  insure 
all  profiles  were  realistic,  and  in  a  small  percentage  of  cases,  replaced  with  new 
randomly  generated  profiles  (for  example,  a  profile  showing  a  master-level  5VM 
where  performance  was  a  7  on  a  7-point  scale  and  leadership  was  a  1  was 
deemed  virtually  impossible  and  so  was  replaced).  Each  5V1VI  profile,  then, 
provides  summary  data  regarding  accomplishments  on  each  of  the  5  vectors. 
These  summary  scores  were  characterized  as  values  on  a  7-point  scale  (1  =  low; 

7  =  high). 


During  the  policy  capturing  workshops,  participants  were  presented  with  these 
120  mock  profiles  that  represented  a  snapshot  of  Sailors'  5VM 
accomplishments /  performance.  Different  workshops  focused  on  advancement 
across  different  skill  levels  (i.e.,  recruit  to  apprentice,  apprentice  to  journeyman, 
journeyman  to  master),  but  the  120  profiles  were  the  same  for  all  workshops. 

The  task  of  the  participants  was  to  review  each  profile,  consider  how  that 
individual's  score  on  all  of  the  individual  vectors  together  contribute  to  an 
assessment  of  the  Enlisted  member's  overall  promotability,  and  then  rate  their 
overall  level  of  promotability  using  a  7-point  scale.  Participants  were  told  that 
when  using  the  7-point  promotability  scale,  it  might  be  helpful  to  apply  the 
following  rule-of-thumb: 

6-7:  outstanding  accomplishments;  definitely  promotable 

3-4-5:  average  level  of  accomplishment;  consider  promoting 

1-2:  below  average  accomplishments;  should  not  be  promoted  at  this  time. 


A  sample  profile  is  presented  below. 


Sample  Profile 

Vector 

Vector  Ratings 

Mean 

1 

2 

3 

4 

5 

6 

7 

Professional  Development 

5.0 

_ 

_ 

[ _ 

_ 

— 

Personal  Development 

5.9 

_ 

_ 

1 - -  . 

1- 

Leadership 

5.6 

1  -  — 

LI _ 

Certifications/Qualifications 

6.2 

1  _  -.1  ^ 

1 

1 

1 

■  / 

Performance 

4.9 

L-  -- 

^ 

■ 

1 

1 

Overall  Promotability  for  Sample  Profile  (Please  circle  your  choice):  1  2  3  4  5  6  7 

In  addition,  to  ensure  that  all  raters  had  a  common  frame-of-reference  regarding 
what  competency  components  defined  each  of  the  five  vectors,  we  provided 
participants  with  definitions  of  the  vectors.  We  also  explained  to  the  participants 
that  they  should  consider  the  5V1VI  profile  scores  to  represent  summary  scores 
generated  from  currently  available  data.  Specifically,  they  were  told  to  assume 
that  professional  development,  personal  development,  and  leadership  scores 
were  produced  from  advancement  examination  data  (i.e.,  the  content  of  the  test 
items  reflects  the  concepts  contained  in  the  vector  definitions).  In  addition, 
certification/  qualification  scores  reflected  physical  fitness  assessment  (PEA)  data, 
and  performance  vector  scores  were  produced  from  past  performance  evaluation 
reports. 


Recall  in  the  preliminary  study,  the  5VM  was  defined  in  broad  conceptual  terms 
and  that  the  objective  for  the  current  study  was  a  more  narrow  focus  on  currently 
available  data  that  could  be  drawn  upon  to  represent  each  vector.  These  new 
vector  definitions  are  provided  in  Table  3. 


Table  3.  Vector  Definitions 


Professional  Development:  This  vector  focuses  on  a  Sailor's  ability  to  acquire  technical  knowledge  and 
skills  required  to  perform  duties  for  a  specific  rating  and  paygrade  through  such  sources  as  formal 
schools,  correspondence  courses  and  on-the-job  training.  Professional  Development  includes 
correspondence  courses,  warfare  designators  and  command  and  platform-specific  training,  such  as  3M 
and  QA. 

Personal  Development:  This  vector  focuses  attention  on  an  individual's  development  as  a  person,  not 
just  as  a  Sailor.  Personal  Development  includes  areas  such  as  personal  financial  management,  health 
and  wellness,  and  safety/operational  risk  management  (ORM).  It  also  involves  educational  activities 
associated  with  fulfilling  higher  education  degree  requirements. 

Leadership;  This  vector  focuses  on  providing  Sailors  with  the  tools  and  critical  thinking  skills  they  will 
need  to  be  effective  leaders.  Components  of  the  Leadership  vector  include:  communication  (e.g.,  conflict 
management,  watchstanding),  professionalism  (e.g.,  military  bearing,  etiquette), 
responsibility/authority/accountability  (e.g.,  rules/regulations,  military  justice),  subordinate  development 
(mentoring.  Navy  benefits),  and  leadership  awareness  (principles.  Naval  heritage). 

Certifications  and  Qualifications;  This  vector  focuses  on  unit-level  requirements  and  related  industry 
certifications  that  are  directly  tied  to  job  proficiencies.  The  Certifications  and  Qualifications  vector  also 
includes  an  individual's  physical  fitness  assessment  (PFA). 

Performance:  This  vector  focuses  on  an  individual's  overall  job  performance.  The  Performance  vector 
takes  into  account  all  issues  related  to  job  performance,  including  task  proficiency  and  productivity, 
adaptability,  initiative,  knowledge  and  support  of  unit/command  objectives,  problem  solving  and  decision 
making,  communication  skills  and  work  ethic.  It  is  the  measurement  of  a  Sailor's  overall  on-the-job 
performance. 


Data  Collection 

In  order  to  gather  a  representative  sample  of  Enlisted  personnel  from  across  the 
U.  S.  Navy,  we  conducted  workshops  at  Bremerton,  WA  -  Submarine  Group  9  (5 
workshops);  Millington,  TN  -  Naval  Support  Activity  Mid-South  (6  workshops); 
Norfolk,  VA  -  the  USS  Eisenhower  Strike  Group  (8  workshops);  and  Pensacola, 
EL  -  Naval  Air  Station  (5  workshops).  In  all,  24  workshops  were  conducted  to 
capture  the  policies  of  the  participants,  using  the  protocol  just  described. 

Separate  workshops  were  conducted  for  three  levels  of  advancement  for  Enlisted 
personnel:  recruit  to  apprentice;  apprentice  to  journeyman;  and  journeyman  to 
master.  To  make  the  task  more  concrete,  we  provided  participants  with 
representative  ranks  for  each  of  the  three  categories.  They  were  recruit  to 
apprentice  (E-3  to  E-4);  apprentice  to  journeyman  (E-5  to  E-6);  and  journeyman  to 
master  (E-7  to  E-8). 


As  can  be  seen  in  Table  4, 129  E-5  to  E-7s  rated  the  120  recruit-to-apprentice 
profiles;  128  E-7  to  E-9s  rated  apprentice-to-journeyman  profiles,  and  61  E-8  to  E- 
9s  rated  journeyman-to-master  5V1VI  profiles.  These  senior  Enlisted  personnel 
represented  aviation,  surface,  submarine,  and  shore-based  work  activities. 

Table  4.  Demographics  for  Enlisted  Level  Workshops 

Apprentice- 

Recruit-Apprentice  Journeyman  Journeyman-Master 


N 

Percent 

N 

Percent 

N 

Percent 

Rate 

E-5 

2 

1.6 

0 

0.0 

0 

0.0 

E-6 

125 

96.7 

0 

0.0 

0 

0.0 

E-7 

2 

1.6 

99 

77.3 

0 

0.0 

E-8 

0 

0.0 

27 

21.1 

10 

16.4 

E-9 

0 

0.0 

2 

1.6 

51 

83.6 

Work  Activity 

Aviation 

8 

10 

4 

Surface  Force 

55 

39 

12 

Submarine 

27 

30 

16 

Shore-based 

37 

30 

18 

Other 

14 

30 

17 

Gender 

Male 

109 

84.5 

117 

91.4 

56 

91.8 

Female 

20 

15.5 

11 

8.6 

5 

8.2 

Ethnic  Origin 

American  Indian 

0 

0.0 

1 

0.8 

0 

0.0 

Asian 

4 

3.1 

0 

0.0 

1 

1.6 

Black 

33 

6.0 

27 

21.1 

7 

11.5 

Pacific  Islander 

0 

0.0 

0 

0.0 

1 

1.6 

Spanish/Hispanic 

6 

4.7 

9 

7.0 

1 

1.6 

White 

84 

65.1 

87 

68.0 

51 

83.6 

Other 

2 

1.6 

4 

3.1 

0 

0.0 

Note.  Totals  may  not  add  to  100  percent  due  to  missing  data.  Totals  for  work  activities  are  inflated  as  a  result 
of  some  respondents  marking  more  than  one  current  activity;  therefore  percentages  are  not  computed. 


Analyses 


Essentially,  the  policy  capturing  methodology  is  a  general  procedure  designed  to 
describe  statistically  the  unique  information  processing  strategies  or  behaviors  of 
individual  raters.  Multiple  regression  analysis  is  used  to  calculate  the  extent  to 
which  overall  ratings  are  predictable  given  scores  on  separate  dimensions  or 
components  (in  the  current  situation,  vectors),  and  the  relative  importance  of 
each  component  in  determining  overall  ratings  (Naylor  &  Wherry,  1965).  Thus, 
the  policy  capturing  analyses  provide  estimates  of  each  participant's  weights  for 
each  of  the  five  vectors. 

In  the  usual  policy  capturing  paradigm,  multiple  regression  is  employed  as  the 
analysis  method.  However,  multiple  regression  can  only  provide  importance 
weights  for  the  independent  variables  (i.e.,  vectors)  when  these  variables 
correlate  zero  with  each  other  (Darlington,  1968).  In  the  current  application,  some 
of  the  between  vector  correlations  were  low  but  non-zero  because  of  some 
adjustments  made  to  the  scores  on  the  profiles  in  order  to  improve  5VM  realism. 
Accordingly,  we  used  a  method  developed  by  Johnson  (2000)  that  provides 
interpretable  relative  importance  weights  under  conditions  of  low,  non-zero 
correlations  between  the  independent  variables. 

These  resulting  weights  can  be  interpreted  as  the  importance  the  participant 
believes  should  be  given  to  each  vector  in  making  advancement  decisions  at  that 
skill  level.  The  analyses  also  provided  an  index  of  consistency  of  policy  for  each 
participant  (R2,  or  variance  accounted  for),  thereby  serving  as  a  data  screen  for 
raters  with  inconsistent  policies.  In  general  terms,  a  rater's  value  indicates 
whether  the  information  provided  in  the  5VM  profiles  was  being  used,  and  used 
consistently  when  making  promotability  ratings.  R^  values  range  between  0.0 
and  1.0,  with  lower  values  identifying  raters  who  failed  to  establish  a  consistent 
strategy  for  using  the  information  to  make  their  ratings.  Any  rater  whose  R^ 
value  fell  below  .55  was  determined  to  have  applied  no  consistent  strategy  for 
how  they  weighted  the  five  vectors.  In  all,  29  of  the  347  participants  were  found 
to  be  inconsistent  in  their  rating  policies,  and  these  cases  were  dropped  from 
subsequent  analyses. 


Results 


Table  5  presents  the  pooled,  summary  results  of  the  policy  capturing  study  for 
Enlisted  advancement.  As  the  table  indicates,  job  performance  was  the  most 
important  factor  for  advancement  from  recruit  to  apprentice  level,  followed  by 
professional  development,  leadership,  and  certifications/qualifications.  For 
advancement  from  apprentice  to  journeyman,  performance  was  still  the  most 
important  factor,  but  leadership  increased  considerably  in  importance,  and  the 
weight  for  professional  development  decreased,  to  a  point  where  the  importance 
of  leadership  and  professional  development  were  comparable.  Certifications  and 
qualifications  still  carries  some  weight,  but  dropped  slightly.  Finally,  regarding 
advancement  from  journeyman  to  master,  participants  weighted  leadership  and 
performance  about  equal  in  importance,  with  the  rest  of  the  vectors  accounting 
for  less  than  23%  of  the  overall  importance  weight.  Across  all  skill  levels, 
personal  development  carries  relatively  lesser  weight. 


Table  5.  Relative  Weights  for  Enlisted  Personnel  Advancement 

Vector 

Recruit  to  Apprentice 
(Percent) 

Apprentice  to 
Journeyman 
(Percent) 

Journeyman  to 
Master  (Percent) 

Professional  Development 

29.68 

22.30 

15.16 

Personal  Development 

7.65 

7.31 

4.04 

Leadership 

16.54 

21.66 

38.51 

Certifications/Qualifications 

11.96 

10.13 

7.62 

Performance 

34.17 

38.60 

34.67 

A  second  objective  of  the  present  study  was  to  compare  the  results  of  the 
feasibility  study  to  the  current  study.  Table  6  provides  a  side-by-side  comparison 
of  the  vector  weights  for  each  study.  As  can  be  seen,  the  general  pattern  of 
importance  weights  is  quite  similar  across  the  two  studies.  Performance 
maintains  a  relatively  stable  and  influential  position  in  the  5VM.  Professional 
development  declines  across  the  course  of  a  career,  and  leadership  increases. 
Certifications  and  qualifications  and  personal  development  play  minor  roles 
throughout  the  career,  although  certifications  and  qualifications  does  carry  an 
importance  weight  of  around  10%. 


While  patterns  of  results  across  the  two  studies  were  quite  similar,  we  also 
examined  whether  the  relative  percentages  within  each  of  the  five  vectors  were 


statistically  different  across  the  two  studies.  Asterisks  in  Table  6  indicate 
significant  differences.  In  all,  7  of  the  15  comparisons  were  found  to  differ  across 
the  two  studies. 


Table  6.  Study  1  and  Study  2  Relative  Weights  for  Enlisted  Personnel  Advancement 


Recruit  to  Apprentice 

Apprentice  to 
Journeyman 

Journeyman  to 
Master 

Vector 

study 

1 

Study 

2 

% 

change 

study 

1 

Study 

2 

% 

change 

study 

1 

Study 

2 

% 

change 

Professional  Development 

17.78 

29.68 

-11.90* 

10.62 

22.30 

-11.68* 

9.33 

15.16 

-5.83* 

Personal  Development 

1.98 

7.65 

-5.67* 

3.47 

7.31 

-3.84* 

2.60 

4.04 

-1.44 

Leadership 

12.26 

16.54 

-4.28 

31.26 

21.66 

-1-9.60* 

41.23 

38.51 

-1-2.72 

Certifications/Qualifications 

9.99 

11.96 

-1.98 

10.05 

10.13 

-0.08 

6.09 

7.62 

-1-2.72 

Performance 

57.99 

34.17 

-1-23.82* 

44.60 

38.60 

-^6.00 

40.75 

34.67 

-1-6.08 

*  Denotes  significant  differences  between  Study  1  and  Study  2  relative  percentages,  using  t-tests  for  differences 
between  independent  samples. 


Discussion 


The  policy  capturing  research  described  here  provided  a  scientifically  sound 
approach  for  pooling  the  judgment  and  wisdom  of  experienced  Enlisted 
personnel  regarding  the  relative  weights  that  should  be  placed  on  each  of  the 
5VM  vectors  in  making  advancement  decisions.  This  study  provided  a  way  for 
the  Fleet  to  collectively  give  us  their  judgment  about  advancement  policy  in  the 
U.S.  Navy. 

The  basic  finding  was  that  job  performance  is  overall  the  most  important  factor, 
but  as  skill  level  progresses,  leadership  becomes  increasingly  important,  to  a 
point  where  performance  and  leadership  are  roughly  equally  important. 
Professional  development  is  also  an  important  factor,  especially  early  in  a  Navy 
career,  and,  certifications/ qualifications  have  some  importance  at  the  lower  and 
middle  advancement  levels.  Finally,  personal  development  was  afforded  lesser 
importance  for  the  advancement  of  Enlisted  personnel. 

While  the  general  pattern  of  weights  in  the  current  study  was  similar  to  those  in 
the  feasibility  study,  differences  between  the  two  studies  were  found.  Most 
noteworthy,  professional  development  received  higher  relative  weights  across  all 
three  skill  levels.  A  second  notable  difference  is  the  elevated  levels  for  personal 
development  during  the  second  study.  Possibly,  knowledge  that  the  summary 
scores  for  professional  and  personal  development  were  a  product  of  the 
advancement  exam  might  have  increased  the  raters'  perception  of  the  concrete 
nature  of  the  vectors,  and  led  to  increased  weights  to  some  extent.  Still,  the 
percentages  for  the  personal  development  vector  were  not  particularly  high 
relative  to  the  other  vectors. 

These  policy  capturing  results  can  form  the  basis  of  an  advancement  algorithm 
that  captures  and  then  operationalizes  the  Fleet's  view  of  advancement  policy 
within  the  5VM.  To  realize  this  Fleet  vision  of  advancement,  each  vector  must 
identify  or  develop  indicators  of  successful  performance,  as  well  as 
accompanying  metrics,  to  score  individuals  on  the  vector  with  enough 
granularity  to  provide  variability  amongst  their  peer  group.  When  these  scoring 
systems  are  developed  within  each  vector,  overall  advancement  scores  can  be 
computed  using  the  policy  capturing  results.  Specifically,  if  a  Sailor  is,  for 
example,  being  considered  for  advancement  from  apprentice  to  journeyman,  the 
scores  within  a  vector  are  weighted  by  the  policy  capturing  weights.  Thus, 
his/her  professional  development  score  is  weighted  by  a  factor  of  22.30,  his/her 
leadership  score  by  a  factor  of  21.66,  and  so  on.  This  approach  fully  utilizes  the 
individual  vectors'  scoring  systems,  but  the  overall  advancement  score  is 
computed  using  the  policy  weights. 


In  sum,  the  policy  capturing  work  provides  a  framework  for  advancement 
algorithms  that  will  reflect  the  Fleet's  conception  of  what  is  important  for 
promotion  at  each  level  of  advancement.  Endorsement  by  policy  makers  of  this 
advancement  approach  will  allow  the  Navy  to  take  the  next  step  toward  a 
human  resource  system  that  embraces  the  5VM  philosophy.  The  specific 
advancement  algorithms  and  scoring  systems  are  now  ready  to  be  built.  Once 
policy  has  been  established,  indicators  of  success  and  equitable  scoring  systems 
within  each  vector  need  to  be  identified  or  developed.  The  resulting  5VM 
advancement  system  will  be  merit  based,  will  represent  the  Fleet's  values  of 
individual  effectiveness,  and  will  be  consistent  with  the  Navy's  human  resource 
management  strategy. 
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